What is claimed is: 



1 LA method for optimizing a solution set for a problem 

2 comprising the sequential steps of: 

3 a) generating a first set of solutions comprising a plurality of 

4 members; 

5 b) creating a second set of solutions using a selection 

6 operator to select members from said first set of solutions; 

X 7 c ) creating a probabilistic model fitting said second set of 

8 solutions, said probabilistic model allowing for multiple variables to be merged 

9 together to form a single new variable whereby hierarchically decomposable 
N 10 problems can be modeled; 

fa 1 1 d) using said model to generate a third set of solutions; and, 

J7 12 e) replacing at least a portion of said first set of solutions 

jjf 13 with said third set of solutions to create a new solution set. 

O 1 2. A method for optimizing a solution set as in claim 1 



M= 2 wherein the step of creating a model fitting said second set of solutions 

m 

S 3 comprises creating a plurality of models that fit said second set of solutions, 

m 4 selecting one model from said plurality of models, and using said one model to 

5 create said third set of solutions. 

6 

7 

8 3. A method for optimizing a solution set as in claim 2 

9 wherein the step of selecting one model comprises using a scoring metric to 

10 score each of said models and searching said scored models to determine the 

1 1 highest scoring of said plurality of models. 
12 

13 4. A method for optimizing a solution set as in claim 3 

14 wherein said scoring metric is chosen from the group of scoring metrics 

15 consisting of a minimum description length metric, and the Bayesian-Dirichlet 
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1 metric; and wherein the step of searching all of said scored networks comprises 

2 using a search method chosen from the group consisting of a greedy algorithm 

3 method, a local hill climbing method, a gradient search, a tabu search, and a 

4 simulated annealing method. 
5 

6 5. A method for optimizing a solution set as in claim 2 

7 wherein the step of selecting one model comprises comparing said plurality of 

8 models through a binary relation to define a better than relation between said 

9 models. 
10 

11 6. A method for optimizing a solution set as in claim 1 

12 wherein the step of creating a probabilistic model comprises creating a model 

13 that models dependencies, independencies, conditional dependencies, and 

14 conditional independencies on a plurality of hierarchical levels between at least 

15 a portion of said merged variables. 
16 

17 7. A method for optimizing a solution set as in claim 1 

18 wherein the step of creating a probabilistic model fitting said second set of 

19 solutions comprises creating one or more models chosen from the group 

20 consisting of Bayesian networks with local structures, and Huffman networks. 
21 

22 8. A method for optimizing a solution set as in claim 1 

23 wherein the step of creating a probabilistic model comprises creating a 

24 probabilistic model that utilizes local structures to represent conditional 

25 probabilities between variables. 
26 

27 9. A method for optimizing a solution set as in claim 1 

28 wherein the step of creating a model fitting said second set of solutions 

29 comprises creating a Bayesian network model comprising one or more features 

30 to model a hierarchal problem decomposition, said one or more features 
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1 selected from the group consisting of local structures, hidden variables, and 

2 histograms. 

3 

4 10. A method for optimizing a solution set as in claim 1 

5 wherein the step of creating a model fitting said second set of solutions 

6 comprises creating a Bayesian network model comprising local structures 

7 selected from the group consisting of default tables, decision trees, and 

8 decision graphs. 
9 

10 11. A method for optimizing a solution set as in claim 1 

1 1 wherein the step of creating a probabilistic model comprises building a network 

12 to model a probability distribution of said second set of solutions, said network 

13 representing a plurality of merged variables as a single variable, said network 

14 representing dependencies and independencies between at least a portion of 

1 5 said merged variables over a plurality of hierarchical levels. 
16 



17 12. A method as in claim 1 1 wherein the step of building said 

18 network comprises building said network using a scoring metric and a search 

19 method to optimize said network. 
20 

21 13. A method for optimizing a solution set as in claim 1 



22 wherein the step of generating a probabilistic model fitting said second set of 

23 solutions comprises generating a plurality of probabilistic models fitting said 

24 second set of solutions, each of said models allowing for multiple variables to 

25 be merged together to form a single new variable whereby hierarchically 

26 decomposable problems can be modeled; and wherein the step of using said 

27 probabilistic model to generate a third set of solutions comprises using each of 

28 said plurality of models to generate a portion of said third set of solutions. 
29 

30 14. A method for optimizing a solution set as in claim 13, 

31 wherein the method further comprises the step of using a first of said plurality 
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1 of models to generate a selected percentage of said third set of solutions, and 

2 using a second of said plurality of solutions to generate a selected percentage of 

3 said third set of solutions. 
4 

5 15. A method for optimizing a solution set as in claim 1 

6 wherein the method further comprises a step of performing niching to preserve 

7 diversity of solutions. 
8 

9 16. A method for optimizing a solution set as in claim 1 

10 wherein said selection operator provides for niching to preserve diversity 

1 1 among said members of said second set. 
12 

13 17. A method for optimizing a solution set as in claim 1 

14 wherein said selection operator comprises a restricted tournament selection 

1 5 operator. 
16 

17 18. A method for optimizing a solution set as in claim 1 

18 wherein said selection operator comprises one or more operators selected from 

19 the group consisting of a tournament selection operator, a truncation selection 

20 operator, a fitness selection operator, a preselection operator, a crowding 

21 selection operator, a clustering selection operator, a lambda-mu selection 

22 operator, and a Boltzmann selection operator. 
23 

24 19. A method for optimizing a solution set as in claim 1 

25 wherein the step of replacing at least a portion of said first solution set with 

26 said third solution set further comprises replacing at least a portion of said first 

27 solution set with said third solution set using a replacement operator that 

28 provides for niching. 
29 

30 20. A method for optimizing a solution set as in claim 19 

31 wherein said replacement operator performs niching using a niching method 
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1 selected from the group of methods consisting of tournament replacement and 

2 crowding. 
3 

4 21. A method for optimizing a solution set as in claim 20 

5 wherein said replacement operator comprises restricted tournament 

6 replacement. 
7 

8 22. A method for optimizing a solution set as in claim 1 

9 wherein the step of replacing at least a portion of said first set of solutions 

10 comprises performing niching to preserve diversity of said solutions through 

1 1 the steps of: 

12 subdividing said first set of solutions into a plurality of sub-sets; 

13 comparing one of said third set of solutions with members of one 

14 of said sub-sets; and 

15 replacing one of said members of said sub-sets with said one of 

16 said third set of solutions if said one of said third solution set is superior to said 

17 members of said one of said sub-sets. 
18 

19 23. A method for optimizing a solution set as in claim 22 

20 wherein each of said sub-sets have between 15-25 members. 
21 

22 24. A method for optimizing a solution set as in claim 22 

23 wherein each of said sub-sets have a number of members greater than 25 and 

24 proportional to the size of the problem being solved. 
25 

26 25. A method for optimizing a solution set as in claim 1 

27 wherein the step of creating a second set of solutions from said first set of 

28 solutions comprises creating a plurality of second sets of solutions, wherein the 

29 step of creating a probabilistic model fitting said second set comprises creating 

30 a probabilistic model fitting each of said second sets, wherein the step of using 

31 said model to generate a third set of solutions comprises using each of said 
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probabilistic models to generate a plurality of third sets of solutions, and 
wherein the step of replacing at least a portion of said first set of solutions with 
said third set comprises replacing at least a portion of said first set with each of 
said third sets of solutions. 

26. A method for optimizing a solution set as in claim 1 
wherein the method further comprises a step of: 

evaluating said new solution set to determine if completion 
criteria are satisfied, and if said completion criteria are not satisfied replacing 
said first set of solutions with said new set of solutions and repeating the steps 
of b) through e) until said completion criteria are satisfied. 

27. A method for optimizing a solution set as in claim 1 
wherein the step of using a selection operator to create a second set of 
members from said first set further comprises using a fitness function to 
determine a fitness for each of said members, and wherein said selection 
operator selects members having a high fitness. 

28. A method for optimizing a solution set as in claim 27 
wherein said fitness function comprises one or more fitness functions selected 
from the group consisting of a co-evolved fitness function, a multi-objective 
fitness function, a multi-modal fitness function, an interactive fitness function, 
an approximate fitness function, a simulation fitness function, and an execution 
fitness function. 

29. A method for optimizing a solution set as in claim 1 
wherein the step of generating a first set of solutions comprises generating said 
first set of solutions using a probability distribution biased by prior knowledge. 
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1 30. A method for optimizing a solution set as in claim 1 

2 wherein said members of said first solution set are coded in a format selected 

3 from the group consisting of fixed and variable length strings. 
4 

5 31. A method for optimizing a solution set as in claim 30 

6 wherein said members of said first solution set are in a format that comprises 

7 one or more formats selected from the group consisting of: binary code, k-ary 

8 code, floating point code, alphanumeric code, real vectors, logical expressions, 

9 program codes, images, random keys, and rules. 
10 

11 32. A method for optimizing a solution set as in claim 1 

H 12 wherein the method further comprises the preliminary step of converting data 

p 13 into said first solution set. 

" 14 

jjr 15 33. A method for optimizing a solution set for a problem 

16 comprising the steps of: 
O 17 generating a first set of solutions comprising a plurality of 

Ls, 18 members; 

% 19 creating a second set of solutions by selecting a sub-set of 

' l; 20 members from said first set of solutions; 

21 creating a probabilistic model fitting said second set of solutions; 

22 using said model to generate a third set of solutions; and, 

23 replacing at least a portion of said first set of solutions with said 

24 third set of solutions to create a new solution set, said replacing comprising 

25 niching to preserve diversity among members of said new solution set. 
26 

27 34. A method for optimizing a solution set as in claim 33 

28 wherein the step of replacing at least a portion of said first set of solutions with 

29 said third set with niching is performed using one or more operators chosen 

30 from the group consisting of a tournament operator, a crowding operator, a pre- 

31 selection operator, a clustering operator, and a fitness sharing operator. 
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2 35. A method for optimizing a solution set as in claim 33 

3 wherein the step of replacing at least a portion of said first set of solutions with 

4 niching comprises the steps of: 

5 subdividing said first set of solutions into a plurality of sub-sets; 

6 comparing one of said third set of solutions with members of one 

7 of said sub-sets; and 

8 replacing one of said members of said sub-sets with said one of 

9 said third set of solutions if said one of said third solution set is superior to said 
10 members of said one of said sub-sets. 

11 

M= 12 36. A method for optimizing a solution set as in claim 33 

S 13 wherein each of said sub-sets have between 15-25 members. 
| 14 

| 15 37. A method for optimizing a solution set as in claim 33 

H 16 wherein the step of creating a second set of solutions comprises using a 

p 17 selection operator to select members from said first solution set, said selection 

^ 18 operator comprises one or more operators chosen from the group consisting of 

\l 19 a tournament selection operator, a crowding operator, a pre-selection operator, 

FP 20 a clustering operator, and a fitness sharing operator. 
21 

22 38. A method for optimizing a solution set for a problem 

23 comprising the sequential steps of: 

24 a) generating a first set of solutions comprising a plurality of 

25 members; 

26 b) creating a second set of solutions using a selection 

27 operator to select a second set of members from said first set of solutions, said 

28 selection operator using a fitness function to evaluate said members of said first 

29 set, said second set members having a high fitness; 

30 c) building a network to model a probability distribution of 

31 said second set of solutions, said network representing a plurality of merged 
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1 variables as a single variable, said network representing dependencies and 

2 independencies between at least two of said merged variables over a plurality 

3 of hierarchical levels; 

4 d) using said model to generate a third set of solutions; 

5 e) replacing at least a portion of said first set of solutions 

6 with said third set of solutions to create a new solution set, said replacing 

7 comprising niching to preserve diversity among members of said new solution 

8 set; and 

9 f) evaluating said new solution set to determine if 

10 completion criteria have been satisfied, and if said completion criteria are not 

1 1 satisfied replacing said first set of solutions with said new set of solutions and 
M= 12 repeating the steps of b) through e) until said completion criteria are satisfied. 

3 13 

y 14 39. A computer program product for optimizing a solution set, 

W 15 the program product comprising computer executable instructions stored on a 

M 16 computer readable medium that when executed by a computer cause the 

p 17 computer to: 

18 a) create a second set of solutions using a selection operator 

- : 19 to select members from a first set of solutions; 

fP 20 b) build a probabilistic model fitting said second set of 

21 solutions, said probabilistic model allowing for multiple variables to be merged 

22 together to form a single new variable, and said model allowing for 

23 relationships to be modeled between at least a portion of said single new 

24 variables over a plurality of hierarchical levels; 

25 c) using said model to generate a third set of solutions; 

26 d) replacing at least a portion of said first set of solutions 

27 with said third set of solutions to create a new solution set; and, 

28 e) evaluating said third set of solutions to determine if 

29 completion criteria have been satisfied. 
30 
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1 40. A computer program product as in claim 39, wherein the 

2 program instructions when executed to cause the computer to build said 

3 probabilistic model cause the computer to build a Bayesian network that 

4 comprises local structures to model dependencies and independencies between 

5 at least a portion of said merged variables. 
6 

7 41. A computer program product as in claim 39 wherein the 

8 program instructions when executed further cause the computer to 

9 communicate with at least a second computer, and to cause the at least a second 
1 0 computer to execute one or more of the program product instructions. 

11 

M 12 42. A computer program product as in claim 41 wherein the 

p 13 program instructions when executed cause the computer to communicate at 

p 14 least a portion of said first set of solutions to the second computer for 

ft 15 processing by the second computer. 

I* 

H 16 

0 17 43. A computer program product as in claim 39 wherein the 
U: 1 8 program instructions when executed cause the computer to communicate with 
% 19 the second computer to communicate instructions to the second computer for 

1 ~ 20 building at least a portion of said probabilistic model. 

21 

22 44. A computer program product as in claim 39 wherein the 

23 program instructions when executed further cause the computer to replace at 

24 least a portion of said first set of solutions with said third set of solutions using 

25 a niching method to preserve diversity of solutions in said new solution set. 
26 

27 45. A computer program product for optimizing a solution set, 

28 the program product comprising computer executable instructions stored on a 

29 computer readable medium that when executed by a computer cause the 

30 computer to: 
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1 a) create a second set of solutions using a selection operator 

2 that selects members from a first set of solutions; 

3 b) build a probabilistic model that fits said second set of 

4 solutions; 

5 c) use said model to generate a third set of solutions; 

6 d) replace at least a portion of said first set of solutions with 

7 said third set of solutions to create a new solution set using niching to preserve 

8 diversity among said new solution set; 

9 e) evaluate said third set of solutions to determine if 

10 completion criteria have been satisfied, and if said completion criteria have not 

11 been satisfied to replace said first set of solutions with said new set and to 
M> 12 repeat steps a)-e). 

S 13 

yi 14 46. A computer program product as in claim 45 wherein the 

[J 15 program instructions when executed cause the computer to replace at least a 

1 6 portion of said first set of solutions with niching further causes the computer to: 
O 17 subdivide said first set of solutions into a plurality of sub-sets; 

1 8 compare one of said third set of solutions with members of one of 

V: 19 said sub-sets; and 

M 

m 20 replace one of said members of said sub-sets with said one of said 

21 third set of solutions if said one of said third solution set is superior to said 

22 members of said one of said sub-sets. 
23 
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